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Abstract 

The storage of binary cycles in continuous-time Hopfield-type networks is 
investigated. We call a cycle defined by a binary matrix £ admissible if a 
connectivity matrix satisfying the cycle's transition conditions exists, and if 
so construct it using the pseudoinverse learning rule. Our main focus is on the 
structural features of admissible cycles and the topology of the corresponding 
networks. We show that £ is admissible if and only if its discrete Fourier 
transform contains exactly r = rank(S) nonzero columns. Based on the 
decomposition of the rows of S into disjoint subsets corresponding to loops, 
where a loop is defined by the set of all cyclic permutations of a row, cycles 
are classified as simple cycles, and separable or inseparable composite cycles. 
Simple cycles contain rows from one loop only, and the network topology is 
a feedforward chain with feedback to one neuron if the loop-vectors in E are 
cyclic permutations of each other. For special cases this topology simplifies 
to a ring with only one feedback. Composite cycles contain rows from at 
least two disjoint loops, and the neurons corresponding to the loop-vectors 
in E from the same loop are identified with a cluster. Networks constructed 
from separable composite cycles decompose into completely isolated clusters. 
For inseparable composite cycles at least two clusters are connected, and the 
cluster-connectivity is related to the intersections of the spaces spanned by 
the loop-vectors of the clusters. 
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1. Introduction 



Applications of artificial neural networks in content addressable (associa- 



fieldl 


1982 


1984; 


Little 


Lopez- Roc 


rfguez 


et al. 



1974 



McEliece et al. 1987; Smia & Orponen 2003 



2005). Hopfield-type networks are among the most 
popular models of artificial neural networks for studying content address- 
able memory. According to Hopfield's original idea, the privileged regime to 
store information has been fixed point attractors, however experiments (e.g., 



Korn & Faure , 2003 ) indicate that cycles are used to store information and 



chaotic dynamics appears as the background regime composed of these cyclic 
"memory bags" . 

In both discrete and continuous asymmetric variants of Hopfield net- 
works, the storage and retrieval of complex sequences including cycles of 



binary patterns have been investigated (Personnaz et al. , 1986 Guyon et al. 



1988 Gencic et al. , 1990), and biologically plausible learning rules such as 



Hebb's rule, the pseudoinverse rule and their variants with or without delays 
have been used. While simulations of networks constructed using Hebbian 
learning rules have been shown to be qualitatively consistent with experi- 
mental recordings 
is well known (e.g. 

do not perform well when the patterns to be stored are correlated which is 
usually the case in practice. Moreover, it has been recently suggested that 
the pseudoinverse learning rule and its variants may take key roles in the as- 
sociative perception of human faces in the human cortex (Zifan et al. 2007) 



(e.g. 


Kleinfeld 


1986; 


Kleinfeld & Sompolinsky 


1988 


), it 


Rojas 


1996 


) that Hopfield networks with Hebbian rules 



and the encoding of location information in the rat hippocampus (Marinaro 



et al. 2007). Therefore, in this and a subsequent paper (Zhang et al. , 2013), 



we study the storage and retrieval of cycles in continuous-time Hopfield-type 
networks using the pseudoinverse learning rule. 



Although the pseudoinverse rule and its variants (Kohonen, 1988 Per- 



sonnaz et al. 1986; Diederich & Opper, 1987) extend to more general cases, 



most investigations in discrete-time Hopfield-type networks characterized or 
were implemented for cycles or sequences of linearly independent patterns 
(e.g. Personnaz et al. , 1986 Kanter & Sompolinsky 1987; Diederich & Op- 



per, 1987). An approach to storing cycles of correlated as well as linearly 



independent patterns in continuous-time Hopfield-type networks has been 
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proposed by Gencic et al. (1990). In this study, a successfully retrieved cy- 
cle is revealed as an attracting limit cycle in the network dynamics, but it 
remains unclear under which conditions the storage and retrieval of a cycle 
is possible. On the one hand, a connectivity matrix in accordance with the 
cycle's transition conditions may not exist, and on the other hand, as shown 



by Zhang et al. (2013), there are cycles of linearly independent patterns 



for which a connectivity matrix can be constructed, however, the dynam- 
ics of the corresponding networks without delay do not show an attracting 
limit cycle. Accordingly, understanding the storage and retrieval of cycles 
in continuous-time Hopfield-type networks requires to address the following 
questions: 

(1) For a given cycle, does there exist a connectivity matrix for a Hopfield- 
type network in accordance with the cycle's transition conditions? If the 
answer to this question is affirmative, we call the cycle admissible and con- 
struct its connectivity matrix using the pseudoinverse learning rule. 

(2) If a cycle is admissible, is it indeed retrieved successfully in the corre- 
sponding continuous-type Hopfield-type network, that is, does the network 
dynamics exhibit an attracting limit cycle that resembles the prescribed tran- 
sitions between the cycle's patterns? 



We address these questions in this and a subsequent paper ( Zhang et al. 



2013). The focus of the present paper is on the first question. We formu- 



late and prove sufficient and necessary conditions for admissibility of cycles, 
and study the topology of the corresponding networks. In the sequel to this 
paper, the bifurcations in continuous-time Hopfield-type networks with and 
without delay are analyzed, which reveals how oscillations corresponding to 
the transitions prescribed by the cycle are created, and which cycles can be 
retrieved successfully. Our main findings are (i) successfully retrieved cycles 
are created in a Hopf bifurcation from the origin, (ii) for networks with ring- 
topology they are destroyed in a saddle-node on limit cycle bifurcation, and 
(iii) most admissible cycles are retrieved in networks with delays for suffi- 
ciently large delay-time, even if they are not retrieved in networks without 
delay. 

In our study, a cycle is defined by a matrix, S, of binary states for the 
firing rates. To store the cycle in a continuous-time Hopfield-type network, 
the connectivity matrix is decomposed into a weighted sum of two compo- 
nents. The first component serves to store the individual states in £ as fixed 
points, which is implemented using the standard pseudoinverse rule. The sec- 
ond, cycle- component serves to implement the transition conditions, where 
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we stipulate that the states in E are traversed in a cyclic manner. These 
conditions lead to a matrix-equation, which has a solution if and only if the 
rows of E span the same space as the cyclic permutations of the rows of E. 
If this condition is satisfied, E is admissible, and we construct a solution of 
the matrix-equation for the cycle-component of the connectivity matrix using 
again the pseudoinverse method. For the discrete-time networks studied by 



Personnaz et al. (1986) and Guyon et al. (1988) only the cycle-component 



is needed to store a cycle, however in the continuous case the fixed point 
component is needed as well. 

Our main objective is to study the structural features of admissible cycles 
E and the topology of the networks constructed from them. Since the storage 
of cycles in the discrete-time networks considered by Personnaz et al. ( 1986 ) 
and Guyon et al. (1988) leads to the same equation for the connectivity 
matrix as for the cycle- component in the continuous case, our results are 
also applicable to discrete networks. A basic result is that if and only if 
the discrete Fourier transform of E contains exactly r = rank(E) nonzero 
columns, then E is admissible. 

Our approach to classify cycles is based on the decomposition of the row 
vectors of E into disjoint subsets corresponding to different loops created by 
cyclic permutations of the rows. If the cycle is admissible, each of these loops 
is associated with an invariant subspace of the row space, U, of E under cyclic 
permutations. This row-decomposition leads naturally to a classification of 
cycles into simple cycles, separable composite cycles, and inseparable com- 
posite cycles. Simple cycles contain rows from a single loop only. Composite 
cycles contain rows from at least two disjoint loops, and for each loop the 
neurons corresponding to the loop vectors in E are identified with a clus- 
ter, which in turn corresponds to an indecomposable invariant subspace of 
U under cyclic permutations if E is admissible. Two clusters are directly 
connected if their subspaces intersect nontrivially, and they are connected if 
they are part of a chain of directly connected clusters. A network constructed 
from a simple admissible cycle has only one cluster, and we show that the 
network topology is a feedforward chain with feedback to one neuron if the 
loop vectors in E are all cyclic permutations of each other. For special simple 
cycles, there is only one feedback and the network topology simplifies to a 
ring. Networks constructed from separable composite cycles decompose into 
completely isolated clusters. 

The paper is organized as follows. In Section 2, the equations for the 
networks and the pseudoinverse learning rule are introduced. In Section 3, 
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the general admissibility criterion in terms of the discrete Fourier transform 
of the cycle matrix is formulated and proved, and the relation of admissible 
cycles with cyclic permutation groups is discussed. Based on the structural 
features of the invariant subspaces of the row space of an admissible cycle, 
in Section 4, admissible cycles are classified into simple cycles, and separable 
and inseparable composite cycles, and for each type of cycles a corresponding 
admissibility condition is derived. In Section 5, we study the topology of 
networks constructed from different types of admissible cycles, and in Section 
6 we discuss some implications of the results presented in this paper. 



2. Hopfield-type Networks and the Pseudoinverse Learning Rule 



A continuous-time Hopfield-type network (Hopfield, 1984) is described by 
a system of ordinary differential equations for ■Uj(i), 1 <i < N, which model 
the membrane potential of the i-th neuron in the network at time t: 

a^ = -^ + ^4^. + j^ = i,2,...,iv, (i) 

where N > 2 is the number of neurons in the network, Cj > and Ri > 
are the neuron amplifier input capacitance and resistance of the i-th neuron 
respectively, Vj(t) models the firing rate of the j-th neuron, and Ii is an ex- 
ternal input fed into the i-th neuron. The connectivity matrix J = (Jij)NxN 
describes the connection strengths of the network. The sign of Jy character- 
izes the polarity of the synaptic connection from the j-th. neuron to the i-th 
neuron. That is, if the output from the j-th neuron excites the i-th neuron, 
then Jij > 0, and if it inhibits, then < 0. 

The firing rate Vj(t) is related to the membrane potential Uj(t) through 
a sigmoid- shaped gain function, Vj = g(uj). Following Hopfield's original 
smoothness assumption, we choose g(u) = tanh(Aw), where the parameter A 
controls the steepness of the sigmoid function. As further simplification to 
([!]) we normalize Ri = Ci = I and neglect external inputs, i.e. h = for 
every i. 

With these simplifications, ([T]) becomes (dots denote time derivatives) 

N 

Ui = —u 



+ Jij tanh(A-Uj). (2) 
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Alternatively, since Uj = arctanh(t>j)/A, (J2]) can be rewritten as a system of 
differential equations for the firing rates, 



N 



A(l - v}) 



JijVj — — arctanh(t>j 



(3) 



Using vector notation, u 
compactly written as 



[Ui, . . .,u N ) 



[Vl, 



v jy) , rt2J) can be more 



ii = — u + J tanh(Au), (4) 

where here and subsequently a scalar function of a vector or matrix denotes 
the vector or matrix obtained by applying the function to each component, 
i.e. 

tanh(Au) = (tanh(Awi), . . . , tanh(AwAr)) T . 

In this paper, we study the structure of binary pattern cycles that can be 
stored in the network modeled by the above autonomous system. Following 
Hopfield (1982, 1984), any iV-dimensional { — 1, l}-valued column vector is 
identified with a binary vector or pattern, and we use + and — to denote 1 and 
— 1. A sequence of q binary patterns is defined by q—1 consecutive transitions, 
£« -> e {2) Z iq \ with = . . . , #>f, #° = + or -, and 

fi = 1,2, ... ,q. A matrix pair (E, F) is called the matrix form of a sequence, 
if E = (£ (1) ,£ (2) , • • • ,£ (p) ), F = / (2) , . . . , / (p) ) and the two matrices are 
related through the transition conditions, £^ — >■ fi = 1,2, . . . ,p. In 
this formulation multiple sequences can be combined, and if terminal states 
are present one has to include the condition £^ — >■ £( g \ so that for each fi a 
transition ^) = ^(^(m)) exis ts. 

According to |Personnaz et al. (1986), the transition conditions can be 

EP, 
dl986fc and 



conveniently formulated in terms of a p x p tra nsition matrix P as F 
where P Ufl = 1 if = £W and otherwise. 



Personnaz et al. 



Guyon et al. (1988) have studied the storage of sequences in discrete-time 



binary networks. In these papers, the transition conditions have been reduced 
to the single matrix equation 

JE = F. (5) 



(1986), that, if FE+E = F, where 



It was pointed out by Personnaz et al 
E+ 
J = 

A sequence is called a cycle of period p, if there exists a smallest positive 
integer p such that for every // G N, ^ +p) = Accordingly, a cycle 



is the Moore-Penrose pseudoinverse of E, then has the exact solution 
FE + , which was called associating learning rule by these authors. 
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Time (t) 

Figure 1: A successfully retrieved cycle E (see text for details) in a network of three 
neurons. A shows the firing rates Vi(t) of the neurons and the raster plot B the overlaps, 
which measure the similarity of the network state v(£) with each of the six patterns in the 
cycle. C illustrates the retrieval of the cycle in the phase space of the system. 



of period p has the matrix form (£,F), where £ = , C > ' ' ' >£ ) an d 
F = (6 > £ j ' ' ' j 6 > £ )> an d the corresponding transition matrix is 



/ 
1 
1 



1 \ 






\0 





1 oy 



(6) 



We are interested in the storage of cycles in the continuous-time Hopfield 
networks defined by Mn. Our approach to compute a connectivity matrix J 



for this purpose follows Gencic et al. (1990). In this paper, the connectivity 
matrix is decomposed as 



Mc j° + CiJ) 



(7) 



where J° serves to stabilize the network in its current memory state and J 
imposes the transitions between the memory states. Here, C\ = 1 — Co and 
Co j < Cq < 1, control the relative contributions of the two components of 
J. The fixed point condition is realized by requiring that v = with a 

parameter < 0\ < 1, is a fixed point if Co = 1. Noting that arctanh(x) is 
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an odd function and |£ 



Jfat 



(p) 



A 



1, this leads, according to (|3), to the condition 
1 



arctanh(/3i£ 



Mh 



A 



arctanh(,$i)£ 



(!->-) 



for every /i, hence JE = /3r-E with (3k ■ 
solution J = (3kJ° with 

J° = SS 



A/3; 



-arctanh(/3i), which has the 



(8) 



Regarding the transition condition, we stipulate that v(£) = fat implies 
v(t + r) = ft/M for some r, and require accordingly for Cq = that 
JE = /3xF. This leads to equation (JsJ) with J = J, which in terms of the 
transition matrix P, equation ([6]), becomes 



JE = EP. 



(9) 



According to the associating learning rule of Personnaz et al. (1986), (JoJ) has 
the solution 

J = EPE+, (10) 

provided that EPE + E = EP. If this condition is not satisfied, ^ has no 
solution. 

The main objective of this paper is the study of the existence and prop- 
erties of the solutions of ^ along with the structural features of the corre- 
sponding cycles, and the network topologies resulting from J° and J defined 
by ^ and (10). A bifurcation analysis and a study of the dynamics of Q 
with ([7]), wit h C and (3 = \(3k > 1 treated as parameters, are given in 
Zhang et al. (2013). In this paper also the extension of Q to a dynamical 
system with a delay, 



u = — u + Cq(3k J° tanh(Au) + C\(3k3 tanh(Au r ) 



(11) 



where r > is the delay-time and u T (t) = u(t — r), is considered. Here 
we show only one example of a successfully retrieved cycle for Q and de- 
vote the remainder of the paper to a mathematical analysis of equation 
(|9]). The example is a network of N = 3 neurons and the cycle consists 
of six states, E = (£« . . . , £ (6) ), with = (+, +, +f, £ (2) = (+, +, -) T , 
£(3) = (+, -) T and £ (3+ ^ = for ^ = 1,2, 3. The retrieval of the cycle 
is illustrated in Figure 1, where in general the overla p of the network wi th the 
u-th pattern is defined as m {v \t) = (t {u) ) T v(t)/N j Zhang et aL~l|2013|). The 
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raster plot of the overlaps in Figure IB as well as the time series in Figure 
1A clearly illustrate that the cycle is retrieved successfully. The parameters 
Co and (3 used in this simulation were Co = 0.6 and (3 = 4. 

In the next section, we derive conditions under which a network for storing 
a given prescribed cycle, i.e. a connectivity matrix J satisfying ^ can be 
constructed, and discuss the structure of the irreducible subspaces of the 
row and column spaces of these cycles under the action of cyclic permutation 
groups. 



3. Admissible Cycles and Cyclic Permutation Groups 

Definition 1. Let P be the cyclic p x p-permutation matrix defined in (jH]). 
A cycle defined by a binary N x p-matrix £ = , C , • • ■ > is s &id to 
be admissible, if there is a real N x N matrix J such that equation ^ is 
satisfied. 

Note that if £ is admissible, the solution to ^ may be not unique. If 



there are several solutions, we select (10) as distinguished solution because 



of its close relationship to J°, see Remark 1(b) below. 



Gencic et al. ( 1990 ) consider a special type of cycles defined by p binary 
vectors E' = (£^,£^ 2 \ ■ ■ ■ , which satisfy the transition condition £^ — > 
£(2) _>.... £(p) _ y —^i 1 ) — y _£( 2 ) £(p) j n this paper we consider 

these cycles as special cases of cycles of period 2p with E = (£', — £'). 



For storing sequences, Personnaz et al. (1986) pointed out that, if the 



associating learning rule FE + E = F is satisfied, the rows of F are linear 
combinations of the rows of E. This follows from the fact that E + E is 
the orthogonal projection matrix onto the subspace of W spanned by the 
rows of E. For storing single cycles, their conclusion can be reformulated 
geometrically as follows: 

Proposition 1. A cycle E of size N x p is admissible, if and only if its row 
space is invariant under P, i.e. 

span{R(E)} = span{R(EP)}, (12) 

where R(E) denotes the set of all row vectors of E. 

Note that |R(S)| < N , and if |R(E)| < N then two or more different rows 
of E are identical, that is, the corresponding neurons traverse the same cycle. 
Although this is a kind of redundancy, we do not exclude this possibility in 
our general discussion. 
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We next formulate a useful alternative admissibility criterion involving 
the eigenspaces of P. Since P is a circulant matrix, P has the orthogonal 
eigenvectors = (l,p k ,p 2k , . . . , p( p_1 ) fc ) T for < k < p, where p = e 2n ^ p 



i = -\/— 1) is the basic primitive p-th root of unity (Lancaster & Tis- 



menetsky, 1985), and in this special case the eigenvalues are p k . We set 
V = (v^\ vPT?. . , v^p-V), Xj = p^ 1 (1 < j < p), A = diag(Ai, . . . , X p ) and 
note that P = VAV^ 1 with V^ 1 = V* /p, where here and subsequently com- 
plex conjugation is marked by an overbar and an asterisk denotes the adjoint 
(complex conjugate transpose) matrix or vector. 

Since the transition matrix P leaves its eigenspaces invariant, it follows 
that if the row space of E coincides with the direct sum of its projections 
onto the eigenspaces of P, then E is admissible. Based on this consideration, 
we obtain the following admissibility criterion. 

Theorem 1. Let E be a cycle whose matrix form is of size N x p, and let 
S = HV. Then E is admissible, if and only if S has precisely r nonzero 
columns, where r = rank(S) = rank(E). 

Proof. Noting that JS = SP = SVAV" 1 implies JS = EA, it follows that 
E is admissible if and only if there exists aniVx iV-matrix J such that 

JE = EA, (13) 

which implies 

Jcol^E) = Aj-col^E), l<j<p, (14) 

where colj(E) denotes the j-th column of E. 

Suppose now that E is admissible and rank(E) = r. Since the columns 
of V consist of eigenvectors associated to distinct eigenvalues of P, it follows 
that E = T,V has r linearly independent columns. Assume col M1 (E), . . . , col Mr (E) 



are linearly independent. Then (14) implies that A M1 , . . . , A Mr are eigenvalues 
of J and col Ml (E), . . . , col Air (E) are the corresponding eigenvectors. If E has 



an additional nonzero column, colj(E), with j ^ {pi, . . . ,/i r }, then by (14) 
this column is an eigenvector of J corresponding to the eigenvalue Xj and 
Xj 7^ A Mi for 1 < i < r. On the other hand, colj(E) is a linear combination of 
col Ml (E), . . . , coLj r (E) which is impossible, since eigenvectors corresponding 
to different eigenvalues are linearly independent. Thus all columns except 
col Ml (E), . . . , coLj r (E) must be zero. 

Conversely, assume E has r nonzero columns col w (E), . . . ,col Mr (E) and 
all other columns of E are zero. Since rank(E) = r, these columns are linearly 
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independent. Let Q be a p x p permutation matrix that maps the column j 
to the column jij for 1 < j < r. Then 



E = 



[0,...,0,col m (E),0,...,0,col Mr (E),0,...,0] 
[col m (E),...,col Mr (E),0,...,0]Q. 



Let E = [col Ml (E), 



, col Mr (E)] and define 



J = EAQ 




Then 



JE 



±AQ T n IrXr rx(p-r) Q 




(p—r)xr "(p—r)x(p—r) 



EAdiag(«i, ...,s p ) 
EA. 



where I rxr is the identity matrix of size r x r, O mxn is zero matrix of size 
m x n, and 



Remark 1. A group theoretical interpretation of admissible cycles E and 
the associated matrices J° and J can be given as follows: 

(a) We denote by Z p the cyclic group of order p defined by addition of integers 
modulo p. Viewed as permutation group, the generator of Z p , addition by 1 
modp, corresponds to the cyclic permutation {0, 1, . . . ,p—l} — > {1, 2, . . . ,p— 
1,0}, and the matrix P is an orthogonal representation of this generator in 
W or C p that cyclically permutes row vectors to the left. Accordingly, the 
matrices P fc , < k < p, form a p-dimensional representation of 7L P with 
P p = J, the p x p identity matrix. 

(b) The admissibility condition span{R(E)} = span{R(EP)} means that the 
rows of E span a subspace of W or C p that is invariant under P, and hence 
under the full representation of Z p , that is, 




l,ifj e 

0, ifj i {fli, . . . , fir} 



□ 



span{R(E)} = span{R(EP fc )}, < k < p. 
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Moreover, with J = EPE+ and EPE+E = EP, we find that J 2 = EP 2 E+ 
and inductively 

jfc = sp fc s + ) o < k < p, (15) 

which shows that 3 P = EE + = J° and J fe E = EP fc for < k < p. This means 
that J restricted to the column space span{C(E)}, where C(E) denote the 
set of column vectors of E, generates a representation of Z p in this subspace 
of M. N . We also note that J° is the orthogonal projection onto span{C(E)}, 
and 

J°J fc = J fc J° = J fc , (16) 
for all < k < p, which is a straightforward consequence of the basic prop- 



erties EE + E = E and E + EE + = E + of the pseudoinverse and (15). Clearly, 
the ranks of J, J° and E coincide and are equal to the dimensions of the 
vector spaces span{C(E)} and span{R(E)} in which Z p acts with matrix 
generators J and P, respectively 

(c) The group Z p has exactly p irreducible complex representations which 
are all one-dimensional and are generated by multiplication of a complex 



number by p , < k < p (Miller, 1972). When restricted to real spaces and 



p ^ R, the multiplications by p and p = Jf~ can be combined to form a 
two-dimensional real irreducible representation space, in which the generator 
of Z p acts by rotation of vectors by the angle 2irk/p. For the representation 
of Z p in the full space of p-dimensional row vectors generated by P, the rows 
in V* are (complex) basis vectors for these irreducible subspaces, and those 
basis vectors with eigenvalues p k for which E has a nonzero column span 
the irreducible subspaces in span{R(E)}. (Real bases in case of p k ^ R are 
obtained by taking real and imaginary parts of these vectors, but we prefer 
to use the complex basis vectors.) Likewise, the non-zero columns of E form 
complex bases of the irreducible subspaces of the Z p -representation generated 
by J in span{C(E)}. We note that, given a row- vector x G C p , xV is the 
discrete Fourier transform of x, and the components of xV are the expansion 
coefficients of x represented by the basis vectors in V*/p. 

Closely related to the group Z p are the p-th roots of unity, which in turn 
are intimately related to the cyclotomic polynomials. In Sections 3-5 we 
will make some use of these polynomials and therefore summarize their basic 
properties in the appendix. 

Given a row-vector x G MP, the orbit of x under P is defined as the the set 
{x, xP, . . . , xP p ~ 1 }. For generic x G MP, this set is a basis of MP, however the 
set of binary row vectors, x — rj, is finite and the orbit of rj, which we call a 
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loop, may span only a proper subspace of W. In the next section we classify 
(admissible) cycles according to the decomposition of R(S) into sets of rows 
belonging to different loops. To pursue this, we will introduce a concept of 
irreducibility that differs from the standard group-theoretical version above. 

4. Classification of Cycles 

4-1- Simple Cycles 

Definition 2. Let i] = (i] 1 ,i] 2 , . . . , r] p ) be a p-dimensional binary row vector. 
The set 

{r)v ■ r)v = ?/P", v = 0, 1, 2, . . . ,p - 1}, 
is called a loop and is denoted by C v . 

Remark 2. For any loop C v , \C n \ < p. More precisely, \C n \ = m, where m 
is a factor of p. In particular, if rj — (+, +,..., +), then \C V \ = 1, as rjP = rj. 

Definition 3. A cycle E is called simple, if its row vectors are from one loop 
generated by some row vector 77, i.e., 

R(S) C C v . (17) 

A cycle S is composite, if it is not simple. 

Definition 4. Let E be a cycle. The set G s = {771,772, • • • , ?7 9 } is said to be 
a generator of S, if 

£ % n£ % =0, V^j, (18) 

and 

<? 

G s = {77!, 772, . . . ,77j C R(S) C (19) 

i=i 

Note that any vector in £ % fl R(S) can be chosen as generator instead of 
r/i in Gs, that is, the generators are unique up to cyclic permutations and 
the condition to be vectors in R(E). In particular, for simple cycles there is 
only one generator, \G%\ = 1, and every row in R(S) can be chosen for this 
generator. A simple criterion for admissibility is the following. 

Proposition 2. A cycle £ is admissible if C v C span{R(S) fl C v } for every 
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Proof. This follows immediately from the fact that, under the given hypoth- 
esis, every row in R(EP) can be represented as linear combination of a subset 
of rows in R(E). □ 

Definition 5. Let 77 be any row vector. The rank of rj is defined as the di- 
mension of the vector space spanned by the row vectors in the loop generated 
by 77, i.e., 

rank(?7) = dimspanj/^}. (20) 

Theorem 2. Let E be a simple cycle generated by rj, i.e., rj G R(S) C C n . 
Then E is admissible, if and only if 

rank(E) = rank(i7). (21) 

Proof. Suppose rank(E) = rank(r/). Then span{R(E)} = span{£^} as R(E) C 
C v . Since P is nonsingular and C V P = C v , it follows that span{R(EP)} = 
span-jX^}, hence E is admissible. Conversely, suppose that E is admissi- 
ble, i.e. span{R(E)} = span{R(EP m )} for all m G N. Assume rank(S) < 
rank(?7). Then there exists 17 G C v with 17 ^ span{R(S)}. Let fj = rjP^ for 
some < n < p. Since 77 G R(S), it follows that 

77 G R(SP^) C span{R(EP^)} = span{R(S)}, 

which contradicts fj ^ span{R(S)}. □ 

Remark 3. In general, although P preserves the rank of any cycle E, the 
vector space spanned by the row vectors of X may not be invariant under 



P. For simple cycles, the condition (21) guarantees that the vector space 



spanned by the rows of E is invariant under P, and hence guarantees the 



admissibility of E. The condition (21) will be referred to as admissibility 
condition for simple cycles. 

4-2. Separable Composite Cycles 

In order to generalize the class of simple admissible cycles to the class of 
separable composite cycles, we first introduce the concept of decomposability 
of the row space of a cycle into irreducible subspaces. 

Definition 6. Let E be an admissible cycle of period p, and let U = 
span{R(E)}. Note that admissibility implies UP = U, i.e. U is invari- 
ant under P. Let V C U be a subspace of U and assume VP = V and 
R(E)HV^0. 
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(a) The subspace V is called reducible if there exists a proper subspace W of 
V such that WP = W and R(E) nW/|. 

(b) The subspace V is said to be decomposable, if V has the direct sum 
decomposition 

V = Vi © V 2 © ■ ■ ■ © V„, (22) 

where n > 2, V, is invariant under P and Vi D R(£) 7^ for every 1 < z < 
n. The subspace V is said to be indecomposable, if it is not decomposable. 



If Vi in (22) is indecomposable for every z, then (22) is called a complete 
decomposition of V. 

(c) The vector space W is called semisimple, if £Y is the direct sum of irre- 
ducible subspaces in the sense of (a). Note that semisimplicity of IA includes 
the case where IA is irreducible, in which case we call U simple. 

Remark 4. We emphasize that, because our purpose is to study the struc- 
ture of the invariant subspaces spanned by the row vectors of S, the concepts 
of reducibility and decomposability introduced in Definition 6 are slightly dif- 
ferent from the standard definitions used in the representation theory of finite 
groups (we require that each subspace contains a row vector of £). An irre- 
ducible / indecomposable invariant subspace in the sense of Definition 6 may 
be reducible/decomposable in terms of the standard definitions of represen- 
tation theory applied to the cyclic group Z p generated by P. 

It is clear that if £ is simple and admissible, then U is simple and con- 
sequently indecomposable, as 77 e Ui implies that rjP k 6 Ui for every k E N. 
However, the converse is not necessarily true. In the next example, we show 
that the vector space spanned by the row vectors of a composite cycle may 
be reducible but not decomposable. 

Example 1. Consider 

/+ + + \ 

+ + --- + 

+ --- + + 

v+ - + - + -J 

and let r]j = row-,(S). Clearly, £ is a composite cycle, as it is generated by 
{771,774}- Let IA = span{R(£)}, U\ = sp&n{£ m } and U2 = span{£^ 4 }. Since 
7/4 = rji — r]2 + 7/3, i.e. 7/4 G IA\, we have that U2 C U\ = U. Moreover, 
R(EP) = {772,773,-771,-774} implies UP = U, hence S is admissible. Since 
both U\ and IA2 are invariant under P it follows that IA is reducible, however, 
IA is not decomposable. 
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Proposition 3. Let E be an admissible cycle with generator = {rji, . . . , r) g } 
Assume U = span{R(£)} is semisimple and let U = U\ © U2 © • • • © U n be a 
decomposition ofU into irreducible subspaces. Then n < q and there exists 
a subset {ii, . . . ,i n } C {1, . . . ,q} such that Uj = span{£ % } for 1 < j < n. 
Moreover, if span{£ T/i } 7^ span{£ T;j } for every i,j G {1, . . . , q} with i 7^ j , 
then n = q. 

Proof. Let i] G Gs and let V G {Uj\l < j < n} be the subspace in the 
decomposition of U that contains r\. Invariance of V implies rjP G V, hence 
r/P 2 g V and by induction C v C V, thus span{£^} C V. Since spanj/^} 
is invariant, spanj/^} D R(£) 7^ and V is irreducible, it follows that V = 
spanj/^}, and there exists no rf G G-%, rj' 7^ rj, such that span{£^/} is 
a proper subspace of span{£ T; } and vice versa. Thus, for i 7^ j, either 
span{£ % } flspan{£ % } = {0} or span{£ % } = spanj/^}. It follows that 
there exists i\, . . . , i n G {1, . . . , q}, ij 7^ ik if j 7^ k, such that 

q n 

span{R(£)} = span{|j£ % } = 0span{£ % .}. 

i=i j=i 



□ 



Example 2. Let % = (+, +, -, +, +, -), r} 2 = —771 and r? 3 = (+, +, +, -, -, -) 
Let S be the 9 x 6-cycle defined by 

where = (77J, ( Vj P) T , (?fcP 2 ) r ) T for j = 1,2,3. Then G s = {771,772,773}, 
span{R(£)} = span{£ m }©span{£ % }, and sp&n{£ m } = span{C m }. Clearly, 
span{£ m } and sp&n{£ m } are irreducible, thus span{R(S)} is semisimple. 
Likewise, for the cycle £ = (Ef,E^), span{R(S)} = span{£ r?i }, i — 1,2, 
hence span{R(S)} is simple. 

In general, the vector space U = span{R(S)} of an arbitrary composite 
cycle £ may have subspaces which are not invariant or do not contain any 
binary row vector of S, or both. By contrast, if U is semisimple, U can be 
decomposed into irreducible subspaces corresponding to the loops of their 
generators, but some of these subspaces may coincide. This coincidence is 
still considered as a degeneracy (see Section 4.3), which we exclude in the 
class of separable cycles introduced next. 



16 



Definition 7. Let E be a composite cycle with generator G-s, i.e. \G^\ > 2. 
We call E separable, if span{R(E)} is semisimple and span{£^} 7^ span{£^} 
for any r],r]' G with rj 7^ rf . If E is not separable, X is said to be 

Note that the hypotheses for a cycle £ to be separable require that W = 
span{R(E)} is invariant under P, i.e. separable cycles are a priori admissible. 

Theorem 3. (Separability Condition for Composite Cycles) LetT, be 
a composite cycle with generator G s = {771, r] 2 , . . . , rj q }. Then E is separable, 
if and only if 



rank(E) = rank fa 



(23) 



Proof. If E is separable, (23) follows directly from Proposition 3 and Defini- 
tion 7. 



Conversely, suppose (23) holds. Since 

9 



span{R(E)} = span{(|J C Vi ) n R(E)} C span{|J C m }, 



i=i 



i=i 



(23) implies that span{£ % fl R(S)} = spaxi{C Vi } for each 1 < i < q, 
spanjX^} n span{£„. } = {0}, if i 7^ j, and hence 



span{R(E)} = 0span{£ (? J. 



i=i 



It follows that each span{£, Vi } is irreducible, thus E is separable according 
to Definitions 6 and 7. □ 

Example 3. Consider 



/ + 


+ 


+ 


+ - - 


- -\ 


+ 


+ 


+ 




- + 


+ 
+ 


+ 




- - + 


+ + 
+ + 


+ 
+ 


+ 




- + + 
+ + - 


- + 


V + 




+ 


- + - 


+ -/ 
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We have that G s = {rji, rj 5 , 777}, where rjj = roWj(E), 1 < j < 7. It is easy 
to see that U\ = spanjX^}, U2 = spanjX^}, and U3 = span{£^ 7 } intersect 
trivially, hence 

U = span(R(E)) =U l ®U 2 ® U 3 , 
which implies that E is separable, and hence admissible. 

4-3. Inseparable Composite Cycles 

By Definition 7, inseparability of a composite cycle E happens in two 
different cases. In the first case, the vector space U = span{R(E)} has 
a reducible but indecomposable invariant subspace, which entirely contains 
another invariant subspace as a subspace (see Example 1). This includes the 
case where U is semisimple and spanj/^} = span{£ T; /} for two different gen- 
erators rj and rj' (see Example 2). In the second case, the vector space U has 
two or more indecomposable (reducible or not) invariant subspaces sharing 
a nontrivial intersection as common proper subspaces. We now discuss the 
admissibility of these two types of inseparable composite cycles. 

Definition 8. Let Gy = {V11V2, ■ ■ ■ ,Vq} be a generator of a cycle E. A 
subset EGy = {ei,e 2 , . . . ,e r }, r < q, of is called an essential generator 
of E, if EGy, is minimal in the sense that 

r 

(a) span{R(E)} C span{ J £ e J; 

i=l 

(b) for any 6j, G EGy, with i 7^ j, span(£ e .)flspan(£ e3 ) is a proper subspace 
of both span(£ e J and span(£ ej ); 

<? 

(c) for every rji G C s , if rji <£ span{ |J £ Vj }, then rji e EG^- 

Note that an admissible cycle may have different sets of essential gen- 
erators, i.e. EGy is in general not unique. Proposition 2 can be directly 
rephrased in terms of essential generators. 

Proposition 4. A cycle E is admissible, if for any essential generator EGy,, 
C t C span{R(E) n C e } for every e G EGy- Conversely, if E is admissible 
and EGy = {ei, • • • , £r} is an essential generator of E ; then 

r 

rank(E) < ^rank(ei) < |R(E)| < AT. (24) 
i=i 
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Remark 5. The condition (b) in Definition 8 includes three cases. 

(a) For every i ^ j, span{£ ei } fl span{£ £j . } = {0}, and q = r. Composite 
cycles in this case are separable. 

(b) For every i ^ j, span{£ £i } fl span{£ ej } = {0}, but q < r. Composite 

cycles in this case are inseparable and degenerate. In this case U may be 

<? 

semisimple or not, and has the complete decomposition U = 0span{£ ei } 

i=i 

(see Proposition 3). Accordingly, we have that a degenerately inseparable 

q 

composite cycle £ is admissible, if and only if rank(S) = ^ rank(ej). This 

i=i 

generalizes separable cycles and includes, for example, the case where for 
some i] G R(S) also — rj G R(S), but C v fl C- v = (see Example 2). 

(c) For some i ^ j, span{£ ei } fl span{£ ej } is a nontrivial proper subspace 
of both invariant subspaces span{£ e .} and span{£ ej }. Composite cycles in 
this case are genuinely inseparable. This type of cycles is more complicated 
than the other two. We next study the structure of this type of cycles, and 
establish an admissibility condition. 

Proposition 5. Let rj and fj be two p- dimensional row vectors. If r] G 
span{£jj} ; then span{£^} C span{£^}. 

v 

Proof. If 7] G Cfj, we are done. Suppose r] G span{£jj}, i.e., rj — ct v rjP v , 

a v G R, but rj i Cfj, then for every t G {1, 2, . . . ,p}, rjP* = (jt a »V pu ^J p * = 

v 

Y (ct u r]P u+t ), i.e., C v C span{£^}, hence span{£ T; } C span{£jj}. □ 

Remark 6. Proposition 5 tells that if a row vector is in the vector space 
spanned by the loop generated by another row vector of the same dimension, 
then the vector space spanned by the loop generated by this row vector 
is a subspace of the vector space spanned by the other one. Since for any 
genuinely inseparable composite cycle, at least two indecomposable invariant 
subspaces intersect nontrivially, it is natural to ask: 

(a) Does there exist a row vector such that this nontrivial intersection is 
spanned by the loop generated by it? 

(b) If this row vector exists, can it be { — 1, 1}- valued? 

As we will see below in Proposition 6, the answer to the first question is 
affirmative, however, it remains unclear whether there always exists a binary 
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row vector such that the loop generated by it spans the nontrivial intersection 
of two indecomposable invariant subspaces. The approach we will use in the 
proof of Proposition 6 only guarantees the existence of a genuine row vector, 
which may or may not be binary. 

Let V be defined as in Theorem 1, i.e. V = (v^°\ v^ 1 ', . . . , v < - p ~ 1 ^), where 
= (1, p\ p 2 \ . . . , p(P" 1 ) fc ) T and p = e 2ni/p . 

Definition 9. A row vector i] is said to annihilate the column v ^ of V, if 
•q V ( k ) = o, i.e. the two vectors are orthogonal. 

Note that, since v ^ is an eigenvector of P and all eigenvalues of P are 
nonzero, r] annihilates if and only if r]P u annihilates for every v G Z. 
We need the following fact about the eigenvectors and eigenvalues of circulant 



matrices, see, e.g., Lancaster & Tismenetsky (1985). 



Lemma 1. Let r\ = (rj 1 , . . . , r] p ) be an arbitrary real and nonzero row vector, 
and let E^ be the p x p-matrix defined by roWj(E^) = r](P T )^~ 1 ' for 1 < 
j < P- Then V*T, n = A V V* , where A v = diag(A^i, . . . , X VtP ) with = 

Extending Definition 2 to non-binary real row vectors and noting that 
R(E^) = C v , an immediate consequence of Lemma 1 is the following: 

Corollary 1. Assume that r/v^ ^ if and only if j G {ki, . . . , k s } C Z p , 
where Z p = {0, 1, 2, . . . ,p — 1}. Then {v^*, . . . ,v^*} is a (complex) basis 
for spanj/^}. 

Proposition 6. Let E be a cycle with essential generator EG^ = {ei, . . . , e r }. 
Assume that for some % ^ j the indecomposable subspaces Ui = span(£ £i ) and 
Uj = span(.£ e .) intersect nontrivially, andUidUj is a proper subspace of both 
Ui and Uj. Then there exists a row vector rj such that Ui C\Uj = span{£^}. 

Proof. Assume that tiV ^ ^ and ejv ^ ^ if and only if k G Ki C Z p and 
k G Kj C Z p , respectively. Assume further that K = KiC\Kj = {ki, . . . , k s }. 
According to Corollary 1, {?/ fcl )*, . . . ,v ( fcs )*} is a basis for Ui f)Uj. Let Pi(x) 
and Pj(x), x G C, be the polynomials Pi(x) = e^x, Pj(x) = e^x, where 
x = (1, x, x 2 , ... , x p_1 ) T . Since the row vectors defined by the coefficients of 
Pi(x) and Pj(x) annihilate exactly the with k G r L p \Ki and A; G Z p \Kj, 
respectively, Pi(x) and contain the minimal polynomials of p k for every 
k G 1i p \Ki and /c G 7* p \Kj, respectively, as factors. Multiplying these factors 
yields a polynomial Pij(x) of degree < p — 1 with Pij{p k ) = for every 
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k G Z p \K and Pij(p k ) 7^ for every k G K. Set P o(x) = Pij(x) if the degree 
of Pij(x) is p — 1. If the degree of Pij(x) is < p — 1, set po(£) = Pij(x)Pij(%), 
where pij(x) is any polynomial such that pij{p k ) ^ for every k & K and the 
degree of po^) is p — 1. Let r/ be the row vector of coefficients of po(x). Then 
i] and T]P U for any z/ G Z annihilate every t> ^ for A; G Z p \K, and ?7t/ fc ) 7^ 
for every k e K, hence span{£^} = span{i/ fcl) *, . . . , v^^*} = Ui HUj. □ 

Example 4. In this example, we demonstrate how to find a row vector as 
claimed in Proposition 6 with the method described in the proof. Consider 
the composite cycle with N = 10 and p = 18 defined by £ = (Xf , E^) T , 
where 



Si = (ef , ( ei P) J , . . . , (e.Py y,Z 2 = (el , (e 2 P)^ , foP^ ) J , 



and 



ei = (+ + + + + + + - + -- -- -- - + -), 

e 2 = (+ + + -- - + + + -- - + + + -- -). 

This is a genuinely inseparable composite cycle with = EG^ = {61,62}. 
The polynomials p\(x) and p%(x) can be factorized as follows, 

Pl (x) = (1 - x )(l + x + x 2 )(l - x + x 2 )(l + x 3 + x 6 )(l + 2x + 2x 2 + x 3 + x 6 ) 
p 2 (x) = (1 - x)(l + x + x 2 )(l - X 3 + X 6 )(l + X 3 + x 6 )(l + x - X 2 ). 

The factors 1 — x, 1 + x + x 2 , 1 + x 3 + x 6 , 1 — x + x 2 and 1 — x 3 + x 6 are 



cyclotomic factors (see e.g. Dummit & Foote (2004)), and the sum of their 



degrees happens to be 17. Multiplying them out gives 

17 
j=0 

thus the row vector 77 constructed in the proof of Proposition 6 is obtained 
as the binary vector with alternating signs, 77 = (+, —,+,—,...,+, — ), and 
span{£jj} = span{r/}. One can easily verify that span{r/} = span{£ ei } fl 
span{£ e2 }. 

Based on their structural features and using a simple inclusion-exclusion 
argument, an admissibility condition for inseparable composite cycles can be 
formulated as follows. 
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Theorem 4. (Admissibility Condition for Inseparable Composite 
Cycles) Let £ be a cycle with essential generator EGy, = {e\, . . . , e r }. Then 
£ is admissible if and only if 

r r 

rank(£) = rank(ej) — Yl dim(span{£ ej } fl span{£ ej }) 

r 

+ Yl dim(span{£ e J n span{£ ej } n span{£ efe }) (25) 

r 

- ■ ■ ■ ± dim( H span{£ e J). 

i=l 



Remark 7. The admissibility condition (25) is valid for any cycle, and the 
conditions (23) and (21) for separability and admissibility of simple cycles, 
respectively, can be thought of as special cases thereof. 

Example 5. Let £ be the cycle from Example 4. One can easily verify that 
rank(ex) = 7 and rank(e 2 ) = 3. Since dim(span{£ ei } fl span{£ e2 }) = 1, this 
cycle is admissible. 



5. Network Topology 

To simplify the discussion, we exclude in this section multiple appearances 
of a binary row vector in a cycle, that is, we consider only cycles £ with 
|R(S)|=iV. 

The classification of cycles £ in Section 4 was based on the decomposition 
of R(£) into subsets of rows associated with disjoint loops. It is, therefore, 
natural to identify the neurons corresponding to the same loop with a cluster. 
However, if a cycle has fewer essential generators than generators, the row 
vectors of a non-essential generator must be combined with one or more 
essential generators and, moreover, there may be several choices for essential 
generators. We therefore make the simplifying assumption that all generators 
are essential generators. For admissible cycles £ this means that for any two 
distinct generators rji, r] 2 , the intersection of their spaces span{£^. fl R(£)}, 
j = 1, 2, is a proper subspace of both of them ({0} if the cycle is separable). 
An immediate consequence of this assumption is that 

q 

rank(£) < ^rank(r/ j ) < N, (26) 

3=1 
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if E is admissible and Gy, = {?7i,- ■ ■ ,Vq}- The clusters are isolated if and 
only if E is separable. If S is inseparable, some of the clusters are connected. 

Regarding the connectivity within a cluster, linear dependences among 
its row vectors will prevent any special structure. We call cycles for which 
such dependences do not occur minimal. 

Definition 10. An admissible cycle E with generator Gs = {^1,^2, • • • ,Vq} 
is minimal, if EGs = G^ and for every 1 < i < q, 

|£ w nR(E)|=rank(^). (27) 

Remark 8. If E is a minimal simple or separable composite cycle, then E is 
of full row rank. If E is a minimal inseparable cycle, then the row vectors in 
R(E) n£ r?i form a basis of span{£ % } for every rji G G^. Thus for any minimal 



cycle E, (26) holds and the inequalities become equalities if and only if E is 
simple or separable. In this case, E has full row rank and E + = E T (EE T ) _1 , 
which implies that J° = EE + = /, the N x N identity matrix. 

For any two (iVxp)-cycles E and E' with R(E) = R(E'), the cycle matrices 
are related to each other by E' = QE, where Q is an iV x iV permutation 
matrix. If in addition E is admissible with connectivity matrix J, then E' 
is also admissible and has connectivity matrix 3' = QJQ -1 . Accordingly, 
if u is the state of the network with connectivity matrix J, then u' = Qu 
is the network state corresponding to J', and solutions u(t) and u'(i) of the 
corresponding differential equations are just permutations of each other as 
tanh(u') = tanh(Q _1 ii) = Q" 1 tanh(u). 

Without loss of generality, we therefore may assume that a minimal cycle 
with generators rji, . . . , r) q has the form 

E = (Ef,E^,...,Ep r , (28) 

where R(Sj) C C Vj , 1 < j < q, and the vectors in Ej are sorted from top 
to bottom as T)jiVjnVj2i ■ ■ ■ > with rj^ = s^^P^, = 1 if — r]j ^ C rjj and 
Sj t G { — 1, 1} if — rjj G C Vj , and < Vj. < Uj k if i < k. We call this form the 
standard form of a minimal cycle. 

The minimality requirement does not suffice in general to induce a special 
network topology within the clusters. We have to require in addition that 
the powers in the Ej are consecutive. 

Definition 11. A minimal cycle E in standard form is said to be a minimal 
consecutive cycle, or briefly MC-cycle, if the powers of P in Ej above are 
consecutive, that is, Vj i = i for all 1 < i < rank(^). 
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In order that Definition 11 is consistent with the minimality requirement, 
the rows in £j must be linearly independent. The next proposition shows 
that this is indeed the case, where for simplicity we consider only the case 

Sa 



J 3i 



Proposition 7. Let rj 7^ be any p- dimensional row vector with rank(r/) = k. 
Then the vectors {77, r/P, r/P 2 , . . . , r]P k ~ 1 } are linearly independent. 

Proof. Let s be the smallest positive integer such that {77, 77P, . . . , 77P S_1 } are 
linearly independent. Then r/P s is a linear combination of {rj, r/P, . . . , riP 8-1 }, 

s-1 

r]P s = ^CM7P". (29) 

Right-multiplying this equation by P yields a representation of 77P S+1 as lin- 
ear combination of {i]P, t]P 2 , . . . , r]P s }, and replacing r]P s in this representa- 



tion by (29 ) shows that t]P s+1 is also a linear combination of {rj, rjP, . . . , r]P s ~ 1 }. 
By induction we find that, for any < v < s — p, r]P s+l/ is a linear combina- 
tion of {rj, r/P, . . . , 77P S-1 }, hence this set is a basis for C v . □ 

For simple MC-cycles there is only one cluster. In Subsection 5.1 we 
discuss the possible connectivity structures in such networks in some detail, 
including the possible values of N for a given p, and we also comment on the 
network topology of simple minimal but non-consecutive cycles. Semisimple 
MC-cycles consist of isolated clusters corresponding to the different loops 
in the cycle. Each of these loops forms a simple MC-cycle, and we just 
give an example in Subsection 5.2. Inseparable minimal (consecutive or non- 
consecutive) cycles are more complicated and will be discussed in Subsection 
5.3. In Subsection 5.4 we demonstrate the effects of fewer essential generators 
than generators by two examples. 

5.1. Simple MC-Cycles 
5.1.1. Network Topology 

According to Definition 11, a simple MC-cycle has the form 

S = (rf , Sl ( V P) T , s 2 ( V Pr, s^P^Tf, (30) 

with rank(?7) = N < p and G { — 1, 1}. Since the image of the last row 
vector of E under P is a linear combination of the row vectors of S, EP has 
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the form SP = AS, where 



/ 




A 



si 






SlS 2 













\ 



0. 
0. 

\ a x a 2 a 3 

with oi, ...,ojv £ K and a\ ^ 0. Moreover, since £ has full row rank, 
S + = S T (SS T )~ 1 , which implies 



SN-ZSN-2 



aAr_l 







(31) 



EPS 



A. 



(32) 



Equations (31) and (32) show that the network constructed from a simple 
MC-cycle consists of a feed-forward chain from the iVth neuron to the first 
neuron, and feedback to the iVth neuron from the subset of the neurons for 
which cii 0, which in any case includes the first neuron. If aj = for 
j > 1, then ax = ±1, and the network topology is that of a ring, with either 
excitatory (a x = 1, J = P T if all Sj = 1) or inhibitory connection (ai = — 1) 
from neuron 1 to neuron N. Vectors of the form rj = (c, a) or (a, —a) have 
rank(?7) < p/2, and if rank(?]) = p/2 = N we have either of these two types 
of ring structures (see Subsection 5.1.3.). 

Example 6. In Figure 2, A and B, we illustrate the topology of the networks 
constructed from the following two simple MC-cycles, 



/+ + + + + + + 



\ 



+ + + + 



and 



/+ + 



+ 



+ 



- - + + + 

- + + + + 
+ + + + + 
+ + + + +/ 



+ - 




+ 




+ 


+ 


- + 


+ 




+ + 




+ J 
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respectively. The cycle S has a "repeating block structure", E = [B,—B], 
where B is the block consisting of the first 7 columns of £ (N — p/2 — 7). 
This causes the image of the last row to be the negative copy of the first row, 
i.e. r/ 7 P = —T)i, where rji = roWj(E). It follows that a± — — 1 and = for 
i > 1, thus the first neuron only sends an inhibitory feedback to the seventh 
neuron (Figure 2A). Similarly, for the cycle E, N = p — 1 = 5, and in this 
case the image of the last row is a linear combination of all other row vectors, 
f)z,P = —fji —f]2 — r\z — ^4 — 775, where fji = roWj(S). Accordingly for this cycle 
dj = —1 for every i, i.e. every neuron sends inhibitory feedback to the fifth 
neuron in the network (Figure 2B). 

The two examples above demonstrate that the value of N — rank(r^) plays 
an important role for the network topology of simple MC-cycles. We discuss 
possible values of N for given cycle-lengths p in the next paragraph. 

For minimal but non-consecutive cycles with iV < p — 1 we can have 
"gaps" in the standard form which lead to feedforward chains interrupted 
by neurons with higher connectivities. The next example demonstrates this 
possibility. 

Example 7. Consider rj = (+, +, +, +, +, +, — , — , — ) (p — 9), and 

S = (rf , ( V Pf, ( V PY, (VPY, (vPY, (vP 6 ) T , (vP 8 ) T f ■ 

This cycle is minimal as rank(^) = rank(S) = 7, but not consecutive. The 
gaps are between the third and fourth rows, and the sixth and seventh rows. 
Since the seventh and first rows are consecutive, there are no other gaps. 
The connectivity matrix is 
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-1 





V 1 



10 





1 




1 



\ 



-1 1 



1 
1 

/ 










-11-11 



and shows that we still have the forward chain 3— > 2 — > 1 — > 7 — > 6 — > 5 — > A, 
but neurons 3 and 6 receive multiple inputs. The network topology is shown 
in Figure 2C. 
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5.1.2. N-p relations 

Definition 12. Let N p : X p — > Q be the function defined by N p (rj) = n if rj 
annihilates (p — n) columns of V, where V is defined as in Theorem 1, X p is 
the set of binary row vectors of length p, and Q — {1, 2, . . . , p — 1}. 

Remark 9. It is a direct consequence of Theorem 1 that N p (rf) = rank(r/). 
Therefore, for a given value of p, the image-set iVp(X p ) contains all possible 



values of N for which there exists rj G X p such that (30) defines a simple 
MC-cycle. Furthermore, in Section 4.3 we have associated with rj G X p 
the polynomial p v (x) = r?(l, x, . . . , x p ~ l ) T , where x is a complex variable. 
Since p r? (p fc ) = r]v^ k \ r/v^ = (i.e. r] annihilates v ^ h ') if and only if Pr,(x) 
has a factor which is a multiple of the minimal polynomial of p k . Thus 
N p (j]) = rank(?7) is intimately related to the factorization of p n {x). 

There appears to be no general characterization of or formula for N p (jj). 
Even for row vectors with repeating block structure such as rj = (er, —a) 
or (a, —a, a) (a G X p / 2 or X p / 3 ), the factorizat ion of p v (x) does not reveal a 
formalizable pattern. We therefore just list the sets N P (W) in Table 1 for 1 < 
p < 20. Note that iV^X 9 ) C A^(X P ) if q divides p, since rank(r/) = rank(cr) if 
r] — (a, a, . . . , a) (p/q repetitions) and a G X 9 . We therefore include in Table 
1 only those values iV G N p (W), for which there exists a row vector fjGX p 
with iV = rank(?7), and 77 is NOT a repetition of some shorter vector a. To 
illustrate how Table 1 was obtained, we compute N p (rj) for a row vector with 
p = 6 in Example 8. 

Remark 10. Vectors 77 of the form 77 = (a, a, ... ) G X p with o G X 9 have 
minimal period < g under cyclic permutations. The number of binary vectors 
of minimal period p is found by subtracting the number of all vectors with 
smaller minimal period from 2 P . An inclusion/exclusion argument shows that 
this number is given by 

s 

M p = 2 P - ^(-l)^ 1 2 p/(Pl i p '2- pi fc ) , 

k=l l<ii<"'<ifc<s 

if Pi, P2, • • • , p s are the distinct prime numbers occurring in the prime factor- 
ization of p (M p = TP — 2 if p is prime). Accordingly, the number of maximal 
loops, i.e. loops with \C V \ = p, is M p /p. 

Example 8. Let rj = (+, +, — , — , — , +). This vector has a repeating block 
structure, rj = (a, —a), where a = (+, +, — ). For p = 6 the matrix V is given 
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Table 1: Values of N e N p (XP) attained by some t) £ F that is NOT of the form 
(a, a, . . . , a) with a G X 9 , q < p, for p < 20. 
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iv 
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iv 


1 
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11 


11 
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12 


6,7,8,9,10,11,12 
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3 


13 


13 
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2,4 


14 


7,13,14 
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15 


11,13,15 
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3,5,6 


16 


8,10,11,12,13,14,15,16 
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7 


17 


17 
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4,6,7,8 


18 


7,9,11,12,13,14,15,16,17,18 
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7,9 


19 


19 


10 


5,9,10 


20 


10,12,13,14,15,16,17,18,19,20 
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1 
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P 2 


-1 


P 4 


P 5 
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P 2 


P A 
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P 2 


P 4 


1 


-1 


1 


-1 


1 


-1 


1 


P 4 


P 2 


1 


P 4 


P 2 


V 1 


P 5 


P 4 


-1 


P 2 


p / 



where p = e w1 ' . The polynomial p v (x) has the following factorization, 

p v (x) = 1 + x — x 2 — x 3 — x 4 + x 5 
= (1 -x 3 )(l +x-x 2 ) 
= (1 -x)(l + x + x 2 )(l +x-x 2 ). 

Since <&i(x) = x — 1 and $3(2;) = x 2 + x + 1 are the first and the third 
cyclotomic polynomials, and p° = 1 is the primitive first root of unity and 
p 2 and p 4 are the primitive third roots of unity, it follows that rj annihilates 
w (2) and Therefore, N 6 (rj) = 6-3 = 3. 
Some of the N- values in N P (W) in Table 1 can be explained directly, 
without factorizing p v (x). We summarize three simple but important facts. 

Proposition 8. (a) {l,p} C N P (K P ) for anyp>2. 

(b) If p > 2 is prime, then N p (K p ) = {l,p}. 

(c) p — 1 G N p (X p ) if p is even and p > 4. 
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Proof. Since rank(+, +,...,+) = 1, it follows that 1 G N p (H p ) for any p. To 
show that p G N p (XP) for p > 2, consider 77 = (— , +,+,...,+) and let be 
the p x p-matrix defined by roWj(S^) = r/(P T )- ?_1 , 1 < j < p. By induction, 
one shows that det(S^) = (— 2) p ~ 1 (p — 2) which completes the proof of (a). 
Statement (b) is an immediate consequence of the fact that & p {x) = 

p 

^2 a?- 7-1 is the minimal polynomial of p k , < k < p, if p is prime and is 

i=l 

irreducible over Q, hence if i] ^ ±(+, +,..., +), and p v {x) cannot 

contain a common factor. 

To show (c), let cr = (+, —,+,—,...,+,—) G X p ~ 2 and set r\ = (a, — , +). 
By performing elementary row operations on the matrix with rows row^S^) 
= 77P l_1 , 1 < i < p — 1, it can be shown that T, v has full rank. The details 
are straightforward but tedious to write down explicitly and will be omitted. 

□ 

Remark 11. (a) If p is prime, then rank(7/) = p for any rj 7^ ±(+, +,...,+). 
For non-prime values of p one also can construct several different vectors 
with rank(r/) = p. For example, if p is odd, then rank(r^) = p if 77 = 
(+, —,+,—,...,+, — , +), which is easily shown using elementary row op- 
erations. A generalization is provided by vectors 77 with Ylv 1 = 1- All our 

i 

case studies indicate that these vectors have rank(r/) = p as well, 
(b) For even p > 4, the vector constructed in the proof of Proposition 
8(c) is just one example of a vector with rank(r/) = p — 1. In general, if 
i] = (77 1 , . . . , rj p ) and V 1 = 0) then i] is orthogonal to (+, +,...,+), and 

i 

rank (77) < p — 1. Case studies indicate that such a vector has maximal rank 
p — 1 if it does not have a "repeating block structure" . 

5.1.3. Simple anti- symmetric cycles 

The characteristics of the cycles considered by Gencic et al. (1990) are 
that the cycle length p is even and the second p/2 columns of the cycle matrix 
are the negatives of the first p/2 columns in the same order. We call such 
cycles anti- symmetric. Here we discuss the possible values of the rank of the 
cycle matrix if these cycles are simple and admissible. 

Proposition 9. Assume p is even, p = 2n, and t] = (a, —a) with a = 
(a 1 , . . . , a n ) G X n . Then we have the following possibilities for rank(r/). 
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Figure 2: Topology of networks constructed, respectively, by a simple MC-cycle with 
N = p/2 = 7 (A, Example 6), a simple MC-cycle with N = p — 1 = 5 (B, Example 
6), a minimal simple but non-consecutive cycle with N = p — 2 = 7 (C, Example 7), 
a separable MC-cycle (D, Example 10) and a minimal genuinely inseparable composite 
cycle (E, Example 11). In panels A-D, excitatory (inhibitory) synaptic connections are 
labeled with red (blue) lines with arrowhead indicating the direction of the connections. 
In panel E, in order to highlight the clusters in the network, connections within the same 
clusters are labeled with black lines, and connections between neurons in different clusters 
are labeled with dark red lines. Directions and polarities of the connections are not shown 
in E. For all of the 5 networks illustrated in this figure, self-connectivities are not included. 
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(a) 1 < rank(?7) < n. Moreover, d = n — ranker/) is even, and if d > 2 the 
o~i satisfy d linearly independent homogeneous linear equations with integer 
coefficients. 

(b) If p = 2 k , k > 2, then rank(r/) = n. 

(c) If p = 2n with n > 2 prime and a ^ ±(+, —,+,...,+,—, +), then 
rank(^) = n. 

Proof, (a) Let rj = (a,— a), a G X n , and define the p x p-matrix X by 
roWj(E) = 77(P T ) J_1 , 1 < j < p. This matrix is a circulant matrix and 
contains all rows of C v , hence rank(r/) = rank(S). According to the properties 
of circulant matrices, the eigenvalues of £ are of the form 

p n 

X = E^V" 1 = !>' - 'VOP*- 1 , (33) 

1=1 8=1 

where p is any pth root of unity Thus rank(?]) coincides with the number of 



distinct pth roots of unity for which the right-hand side of (33) is nonzero. 
Since r]P n = —77, it follows that rank(r/) < n, and clearly rank(^) > 1, which 
proves the first statement of (a). To complete the proof of (a), we note that 
the 2n distinct roots of x 2n — 1 (x G C) comprise n roots with x n = 1 and 
n roots with x n = —1. Thus d = n — rank(^) coincides with the number of 
distinct roots of x n = — 1 for which 

n 

p a (x) = ^aV" 1 = 0. (34) 

i=l 

If n is odd, p a (— 1) 7^ 0, and if n is even, (— l) n = 1, thus all roots in question 
have nonzero imaginary parts, which implies that d is even. If d > 2, p a (x) is 
divisible by a cyclotomic polynomial $ m (x), where m divides n but not 2n. 
The degree of $ m (x) is given by Euler's totient function, ip(m), and is even. 
The condition that p a (x) factors through $ m (x) then leads to <£>(m) linearly 
independent homogeneous equations that must be satisfied by the a\ and 
since $ m (x) has integer coefficients, the coefficients of these equations can be 
chosen as integers as well. If p a (x) contains several cyclotomic polynomials 
$ m .(x), 1 < j < r, as factors, the number of linear equations satisfied by a 
is ip(mi) + ■ ■ ■ + (p(m r ), and all these relations are linearly independent as 
the cyclotomic polynomials are distinct and irreducible over the rationals. 
(b) If p = 2 k , k > 2, the only factor that divides p but not n = 2 k ~ 1 is p. 
The cyclotomic polynomial of p is § p (x) = 1 + x n and has degree n, that is, 
$p(x) cannot be a factor of p a {x) which has degree n — 1. 
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(c) Assume now that n > 2 is prime. In this case, $2n(^) = $n( — %) is the 
only cyclotomic polynomial in question and is given by $„(— x) = 1 — x + x 2 — 
x 3 + ■ ■ ■ + x n . Thus, if cr is not of the form a = ±(+, —,+,—,...,+, — , +), 
$ n (— x) does not factor through p a {x). □ 

Since for rj = (cr, —a) a "rank deficiency" (rank(^) < n) occurs only 
if cr satisfies a system of linear equations, the number of cr's for which 
rank(?7) = n is considerably larger than the number of cr's for which 77 
has a rank-deficiency. Thus "generically" we expect that vectors of the 
form (a, —a) have full rank n. The vector a = (+, —,+,—,...,+,—, +) 
is, of course, a very special case as (cr, — cr) has the repeating block structure 
(+, —,+,—,...,+, — ) which has minimal rank 1. We illustrate the occur- 
rence of rank deficiencies by an example. 

Example 9. Let p = 18 = 2 • 3 2 , i.e. n — 9. The cyclotomic polynomials 
that can give rise to a rank deficiency are here $g(x) = 1 — x + x 2 and 
$180*0 = I — x 3 + x 6 . The condition that p a (x) = cr 1 + o 2 x + • • • + cr 9 x 8 
factors through &e( x ) leads to the equations 

a l _ a S _ ^4 + a S + a 7 _ a 9 = 0> 
a 2 + a 3 _ a 5 _ a 6 + a 8 + a 9 = Q 

The only binary vector satisfying these conditions (up to cyclic permutations) 
are ±a^\ ±a^ and ±a^ 3 \ where 

= (+,+,+,+,+,+,+,-,+), 
° {2) = (+,+,+,-,+,+,-,-,+), 
^ {3) = (+,+,-,+,+,-,+,-,+). 

Since &q(x) has a single pair of complex conjugate roots, rank(a < -^, — cr^) = 
9 — 2 = 7, v — 1, 2, 3. Similarly, in order that $is(x) factors through 
the conditions cr-? + a^ +3 = for 1 < j < 6 must be satisfied, leading 
to rank(<r, — cr) = 3. All vectors with block structure a = (cr, —a, a) with 
a G X 3 have this property, and lead to 77 = (cr, <r, a) with cr = (a, —5"), i.e. 
rank(?7) = rank(cr). This includes a = (+,+,+) with rank(cr) = 3, and 
a = (+, — , +) with rank(cr) = 1. In the latter case, both $6 0*0 an d 3>is0*0 
are factors of ^(x). 

The rank of generic vectors (without rank deficiency) of the form (cr, cr) 
or (cr, — cr) is equal to the length of cr. The converse question is under which 
circumstances a vector i] G X p with p even and rank(^) = p/2 has this form. 
We state two simple sufficient conditions for this property. 
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Proposition 10. Assume p = 2n, rj G X p ; and rank(^) — n. If p — 2 k , 
k > 2, or n > 2 is prime, then i] is either of the form (a, a) or (a, —a) for 
some a G X n . 

Proof. Let i] = (a, a) with <r, <x G X n . We consider again the matrix E 
defined in the proof of Proposition 9 with eigenvalues 

n 

i=i 

where p is a pth root of unity, p = 2n. Assuming that ranker/) = n, there 
exist precisely n distinct roots p of x 2n = 1 for which A = 0. We decompose 
these roots again into roots satisfying x n = 1 and x n = —1, respectively, and 
set accordingly 

n 

A ± (x) = ^(^±aV" 1 - 
i=i 

(a) Assume that p = 2 k (n = 2 fc ~ 1 ) for k > 2. If there exists a root p of 
x n = — 1 for which A = 0, A (a;) must contain the cyclotomic polynomial 
&2n(x) = 1 + x n as a factor, which is only possible if a % — b % = for all 
i, because X(x) has at most degree n — 1. Thus = (<r, cr) in this case. 
Conversely, assume that all roots p for which A = are roots of x n = 1. Then 
A+(x) must contain all cyclotomic polynomials § u (x) for z/ = 1, 2, . . . ,2 fc ~ x 
as factors. Since the product of these polynomials is 1 — x n , this cannot hold 
unless a 1 + a % = for all i, thus i] = (a, —a) in this case. 

(b) The case p = 2n with n > 1 prime is treated similarly. Here the cy- 
clotomic polynomials which factor through x n — 1 are 1 — x and § n (x) = 
1 + x + ■ ■ ■ + x n_1 , and the cyclotomic polynomials which factor through 
1 + x n are 1 + x and <3> n (— rr). Since n > 1, either $ n (x) is a factor of 
A + (x) or $„(— rr) is a factor of A_(x), which implies that either a = a or 
a = -a. □ 

An extension of Proposition 10 to more general values of p appears highly 
nontrivial, because a multitude of cyclotomic polynomial have to be consid- 
ered if the prime factorization of n is more complicated. We have examined 
all vectors rj with rank(^) = p/2 for p < 20 and found that all these vectors 
have the form (a, a) or (a, —a). Other vectors with rank(^) = p/2 may exist 
for larger values of p, but if so we expect the number of these vectors to be 
much smaller than the number of (<r, a)- or (er, — cr)-vectors of full rank. 
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5.2. Separable MC- Cycles 

For separable MC-cycles with generators Gs = {Vij V2, ■ ■ ■ , Vq}i the spaces 
span-jX^ } and spa.n{£ rik } intersect trivially if j ^ k. If E is in standard form, 
this implies immediately that J has a block structure, J = diag(Ji, . . . , J r ), 
where is an N*. x A^-matrix of the form (31) with Nk = rank^). Ac- 



cordingly, a network constructed from a separable MC-cycle is decomposed 
into r disconnected clusters and for each cluster the connectivity matrix has 
the form corresponding to a simple MC-cycle. 

Example 10. Consider the 7 x 8-cycle 

/+ + + + -- -- ' 

+ + + -- -- + 
+ + -- -- + + 



V + 



- - + 

- + - 



+ 
+ 



+ 



/ 



This cycle has the generators rjx, r/ 5 , rjr (rjj = roWj(E)) and is separable and 
in standard form. Moreover, 774 P = —771, %P = —T)5, and 1] 7 P = —^7. Thus 
the network is decomposed into three clusters consisting of neurons 1, 2, 3, 
4, neurons 5, 6, and neuron 7, with cycle-connectivity matrices 



/ 1 \ 

10 

1 

\ -1 / 



1 
-1 



and —1, respectively. The topology of this network is illustrated in Figure 
2D. We note, however, that the cluster consisting of neuron 7 cannot show 
oscillations without delay, since a ID dynamical system does not have limit 
cycles. By contrast, with delay included, we can find oscillations already for 
N = 1 for appropriate parameter values (Zhang et al. 2013). 



General separable cycles still can be decomposed into isolated clusters 
as span{R(E)} is semisimple, however, the network topology in each cluster 
maybe more complicated (see Subsection 5.4). The issue with separable 
cycles is that, even if the subcycles corresponding to the different clusters are 
retrieved, these oscillations are in general not synchronized. We comment on 
this issue further in Section 6. 
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5.3. Minimal Inseparable Cycles 

For minimal inseparable cycles E with generators Gy, = {^1,^2, • • • ,Vq}i 
at least two subspaces span{£ % flR(E)} and span{£ % flR(E)} (j 7^ k) have 
a nontrivial intersection. Accordingly, E does not have full row-rank, 



rank(E) < rank(?7j) = N, 



i=i 



which implies in particular that J° 7^ I. It is still possible to partition the 
network into clusters, but some clusters may be connected and the network 
topology within the cluster corresponding to the loop C Vj will in general 
not coincide with the network topology predicted by the submatrix Ej of 
the corresponding simple cycle. Thus the consecutiveness requirement does 
not have an effect, whereas the minimality requirement takes care that the 
sub-matrices in J° and J defining the connectivities within the clusters are 
non-singular. The following example illustrates these features. 

Example 11. Consider the 10 x 12-cycle E = (Ef , T% , E3 ) T , where 



>3\T\T 



with 
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2\T\T 



J = 2, 3, 
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This is a minimal inseparable admissible cycle with generator = 
The connectivity matrix J is given by 
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J = 
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and J° has the same block structure as J (with self-feedbacks of all neu- 
rons). From the form of J (and J°) we infer that the cluster corresponding 
to 771 is connected to the clusters corresponding to 772 and 773, while the latter 
two clusters are not directly connected. This connectivity structure is due 
to the fact that span{£ m } intersects span{£^ 2 } and span{£^ 3 } in the one- 
dimensional spaces spanned by (+, —,+,—,...,+, — ) and (+, +,+,..., +), 
respectively, whereas span{£ % } and span{>C^ 3 } intersect trivially. The net- 
work topology for this example is shown in Figure 2E. In general, two clusters 
corresponding to two generators 77,77' G Gs are connected, if there exists a 
sequence 77 = 77!, 772, . . . ,r) s -i,r) s = r( of generators such that span{£ % } and 
span{£ %+1 } intersect nontrivially for < j < s. 

5.4- Further Examples 

The examples in this subsection serve to illustrate the possible effects of 
fewer essential generators than generators. Consider a cycle S with EG^ = 
{ei,...,e r }. If r < the loop vectors of at least one generator are 

contained in the span of the loop vectors of another essential generator. 
Assuming |R(S)| = N, this implies 



and we encounter again a rank-deficiency that will destroy special structures 
in the clusters corresponding to the essential generators. 

Example 12. The 6 x 6-cycle 



has two generators, rowi(E) and rowe(S), but rowe(S) = rowi(E)+row3(S) + 
row 5 (S), thus there is only one essential generator, e = rowi(E). Without the 
sixth row, E would be a simple MC-cycle with ring-topology. The presence 
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of the sixth row destroys this structure, which is revealed in the following 
forms of J° and J, 
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Example 13. The cycle 



has three generators and one essential generator that can be chosen as the 
first or second row. Since S is non-singular, £ is admissible and J° is the 
identity matrix. A successfully retrieved cycle shows three consecutive phases 
1, 2, and 3 during an oscillation. In phases 1 and 2, neuron 1 is "on" (+) 
and in phase 3 it is "off" (— ), while neuron 2 is "on" in phase 1 and "off" in 
phases 2 and 3. Clearly, neuron 3 is "on" during all 3 phases. The matrix J 
is given by 

/-iii 

J = -1 
\ 1 

and shows that neurons 1 and 2 form an excitatory/inhibitory pair, whereas 
neuron 3 acts excitatory on neuron 1. Without this third neuron the oscil- 
lations of neurons 1 and 2 as required by the first two rows of S could not 
be implemented, since the submatrix of S consisting of these rows is not 
admissible. 



6. Discussion and Conclusion 

In this paper we have studied the structural features of admissible cy- 
cles and their relation to the topology of the corresponding networks. Our 
study was pursued in the context of continuous-time Hopfield-type networks, 



but the results apply to the discrete networks considered by Personnaz et al. 



37 



(1986 


) and 


Guyon et al. 


(1988) 



ity matrix in these networks are the same as the conditions imposed on the 
cycle component of the connectivity matrix in the continuous case. In this 
paper we have restricted our study to single cycles; an extension to storing 
multiple cycles is given in (Zhang et al. 2013). 

We have shown that if and only if the discrete Fourier transform E = 
of a cycle matrix E contains exactly r nonzero columns, where r = 
rank(E), then a continuous-time asymmetric Hopfield-type network can be 
constructed with the pseudoinverse learning rule. Based on the structural 
analysis of the invariant subspaces of the row space of E, the admissible 
cycles have been classified into simple cycles, and separable and inseparable 
composite cycles. This classification was based on the decomposition of the 
row space of E into subsets corresponding to disjoint loops. The admissibility 
of a cycle implied that all vectors of a loop are in the row space of E if E 
contains some of these loop vectors. If no loop-space associated with E is 
a subspace of another loop-space (the generators are essential generators), 
we have identified for each loop the neurons associated with the loop vectors 
contained in E with a cluster. For general admissible cycles the clusters are 
connected, and the connectivity of the clusters depends on the intersections of 
their loop-spaces. Two clusters are directly connected if their indecomposable 
invariant subspaces intersect non-trivially. They are "indirectly" connected 
if they are part of a chain of directly connected clusters. 

If an admissible cycle is separable, the clusters are completely isolated. 
In this case each cluster corresponds to a simple cycle associated with a 
generator of E. If the simple cycle is minimal and consecutive, the cluster 
has the form of a feedforward chain from the last neuron to the first neuron 
with feedbacks to the last neuron from the other neurons. If in addition 
the length of the cycle, p, is even and the rank of the generator is p/2, we 
generically find a ring structure with excitatory or inhibitory connection from 
the first neuron to the last neuron, but we cannot exclude that special loops 
with these properties exist for which no ring-structure occurs. If the simple 
cycle is minimal but non-consecutive, we find more than one feedforward 
chains. Regarding non-minimal simple as well as composite cycles, it would 
be interesting to find equivalence relations similar to those of |Golubitsky et 



al. (2005), and Golubitsky h Stewart (2006), relating networks constructed 



from non-minimal cycles to networks constructed from minimal cycles. 

The complete isolation of the clusters of separable cycles means that each 
cluster has its own subcycle. The issue is that we cannot expect the differ- 
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ent subcycles to synchronize, preventing the network to traverse the cycle 
states in the order prescribed by the cycle matrix. In this case an additional 
synchronization mechanism must be introduced to enforce synchrony. Such 
a mechanism can be in the form of a small coupling among the clusters or 



through an external periodic input acting as pacemaker, see Zhang et al 



(2013) 



The generation of cyclic patterns in biological neural networks is associ- 
ated with central pattern generators (CPGs), and the storage and retrieval 
of cyclic patterns are fundamentally important in animal neural systems. 

In animal neural systems, the CPG networks with different topologies 



have been experimentally identified in different animal species (e.g., MacKay 



Lyons, 2002). Moreover, as some animal movements, such as swallowing, 
gastrointestinal motility etc., often require the coordination of several func- 
tional groups of muscles, some CPG networks controlling these muscles can 



reorganize during different phases of the movement (Dickinson & Moulins 



1992 Meyrand et al. 1994). Such CPG networks consist of pools of neurons 
that can function in several CPGs involved in the organization of various 
motor behaviors. 

Our study showed that the topology of networks constructed from an 
admissible cycle is determined by structural features (decomposition into 
invariant indecomposable subspaces) of the cycle matrix. Networks con- 
structed from cycles with different structural features have different network 
topologies. If animal movements change, the structural features of the corre- 
sponding cyclic patterns change as well, which requires reorganization of the 
CPGs. The analysis of these structural features and their implications on 
the network topology may provide a scenario for studying the reorganization 
of CPGs during different phases of movements, when the dynamics of single 
neurons does not play a significant role in generating cycle patterns. 

Appendix A. Cyclotomic Polynomials 

We summarize here the basic properties of the cyclotomic polynomials 



used in Sections 3-5, for details see Dummit & Foote (2004) 



The cyclotomic polynomial of order p is defined by & p (x) = Yl r ( x ~ 
x G C, where the x r encompass all primitive p-th roots of unity, that is, 
x v r = \ and x™ ^ 1 if 1 < n < p. The total number of such primitive 
roots is given by Euler's totient function, ip(p). If p — TljPj 3 with distinct 
primes pj is the prime factorization of p, then ip(p) = YljPT 3 1 (Pj ~ !)• The 
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important property of the cyclotomic polynomial is that they have integer 
coefficients and are irreducible over the rationals. Moreover, <& p (x) is the 
minimal polynomial for each root x r , and the product of all &d{x) for which 
d is a factor of p and 1 < d < p is x p — 1. The only cyclotomic polynomials 
of odd degree are $i(x) = 1 — x and $ 2 (^) = 1 + x, all for p > 2 have 

even degrees as their primitive roots are all complex. Some basic properties 
of &p(x) are: 

v 

= x 1 ' 1 if p is prime, 

i=i 

&2p(%) = &p(— x ) if V is °dd, 
= S s (a^), 

where g is the radical of p, i.e. the product of all distinct prime numbers 
occurring in the prime factorization of p. The last property implies in partic- 
ular &p(x) = 1 + x n if p = In = 2 k , k > 1. The third cyclotomic polynomial 
is $3(0;) = 1 + x + x 2 and has the roots e ±27rl//3 . 
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